1. Filed of the Invention
The present invention relates to an image stabilizing device which is built in a digital camera or the like.
2. Description of the Related Art
An image stabilizing technique is a technique to stabilize (correct) inevitable hand movement caused at the time of photographing a still image. The image stabilizing technique is achieved by detecting the hand movement and stabilizing an image based on the detected result of the hand movement.
Methods for detecting the hand movement include a method using a hand movement sensor (an angular velocity sensor) and an electronic method for detecting the hand movement by analyzing the image. Methods for stabilizing the image include an optical method for stabilizing a lens and an imaging device and an electronic method for removing blur caused by the hand movement, by image processing.
On the other hand, a completely-electronic image stabilizing technique, that is, a technique in which only one photographed blurred image is analyzed and processed so as to create an image on which blurring is removed, has not reached at the level of practical use. In particular, it is difficult to obtain hand movement signals with accuracy obtained by the hand movement sensor by analyzing one blurred image.
Accordingly, it is practical that hand movement is detected by the hand movement sensor and blurring caused by the hand movement is removed by the image processing using the hand movement data. To remove blurring by the image processing is referred to as image restoration. In addition, a technique using the hand movement sensor and the image restoration is referred to as electronic image stabilizing herein.
A technique has been developed, in which data of the hand movement sensor is converted into Point Spread Function (PSF), which is a model representing deterioration of an image, which is caused by the hand movement, an image restoration filter is created from PSF, and an image is restored by means of this image restoration filter.
However, even though the hand movement can be accurately represented by PSF, hand movement stabilizing effects vary depending on a size of the hand movement, a trajectory shape of the hand movement, or the like. According to the size of the hand movement, the trajectory shape of the hand movement or the like, there is a case where the image stabilizing effect is low, or an image quality is adversely deteriorated when compared with that before the image stabilization.
In general, the image restoration filter is configured of a two-dimensional FIR filter. With the limitation of hardware, however, an upper limit of the size is fixed. When the size of the hand movement exceeds a certain constant value, components of the image restoration filter cannot be fit into the number of taps of the FIR filter. Therefore, the stabilizing effect becomes deteriorated. In contrast, when the size of the hand movement is extremely small, sufficient stabilizing effects can be obtained not by using the image restoration filter created from PSF, but even by using an unsharp masking filter.
In addition, even when the size of the hand movement is constant, the stabilizing effects may be changed depending on whether a speed change state of the hand movement is a uniform speed or non-uniform speed (acceleration, deceleration, and acceleration and deceleration). In case of the uniform speed, a specific frequency component is completely lost. Therefore, there exists the frequency component that cannot be restored by the image restoration filter. In addition, the number of the taps of the image restoration filter is prone to be larger when compared with the case of the non-uniform speed. Furthermore, ringing appears largely in the image restored by the image restoration filter. In addition, the stabilizing effects become smaller as the trajectory shape of the hand movement becomes complicated.
It should be noted that it is disclosed in Japanese Patent Laid-open Application No. Hei 7-226905 that detected blurring information is converted into PSF in consideration of an aperture curve of a sector.
In addition, it is disclosed in Japanese Patent Laid-open Application No. Hei 10-215405 that two points on an image are designated with a marker and an image is restored by convolution operation by using inclination and length thereof.
Furthermore, the following technique is disclosed in Japanese Patent Laid-open Application No. Hei 11-24122. That is, an angular velocity is obtained with integration of three periaxial angular accelerations obtained by three angular acceleration sensors. In addition, a change over time for aperture of shutter is detected. With the angular velocity and the aperture change over time, a transfer function (PSF) representing a blurring state at the time of photographing is created. The image is stabilized by performing inverse transformation of the transfer function in relation to the picked-up image.
In addition, it is disclosed in Japanese Patent Laid-open Application No. Hei 11-27574 that a picked-up image is converted into two-dimensional frequency space and a parameter of a direction and a size of hand movement are automatically detected.
Furthermore, it is disclosed in Japanese Patent Laid-open Application No. 2000-298300 that an hand movement vector in each pixel on a subject image is estimated and image stabilization according to the estimated hand movement vector is performed.
In addition, it is disclosed in Japanese Patent Laid-open Application No. 2004-88567 that hand movement information at the time of image pick-up is added to an image data and the image data is stabilized at the time of restoring the image data based on the added hand movement information.